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ABSTRACT 


The effects of detection equipment integration time on 
the optimal evasive trajectory of an isotropic acoustic 
radiator are studied. The boundary cases of infinite and 
zero integration time are examined. The infinite integra- 
tion time case is formulated as a control problem and a 
maximum principle solution is obtained. The results consist 
of advice as to the choice of control vectors. The zero 
integration time problem is formulated in ordinary differ- 
ential equations and the results consist of control vector 
advice. The relative movement plots and control vectors of 


the two bounding cases are compared. 
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1.0 INTRODUCTION 


1.1 THE PROBLEM UNDER CONSIDERATION 

It is the nature of submarine warfare to avoid detec- 
tion. Indeed, one of the major advantages of the submarine 
is surprise. The passive detection of low level acoustic 
signals (in a water environment) is a function of many 
factors. Among these are the power density of the signal 
at the detector and the period of time over which the signal 
is received. This paper seeks to examine the differences 
in suggested evasion tactics which result from a study of 
these two factors. 

The power density received by the detector may be 


represented as shown in Figure 1.1.1. 


Power 
density 
received 


<__ T —__> 


“Time 
t 


Figure 1.1.1 





Due to the nature of the detection equipment, a memory, 
or integration time is required. The integration time is 
shown as T in the figure. For an evader, which is producing 
the noise signal and wishes to minimize the probability of 
detection (all other factors being held constant), the 
objective is to minimize the maximum area over time under 
the power density curve over the integration time. The 
memory requirement of the equipment is the same for each 
instant of time. While time advances this "window" of 
integration time, T, slides along with the power density 


curve. Expressed mathematically the objective is to: 


a 
a) 
Min Max P(u) du 
P t 
T Ci. 1) 
c-5 e 


A solution in the general case is not proposed, only very 
large and very small values of T will be examined. That is, 
the extreme effects of integration time on the optimal 
evasive trajectory are studied. 

Typically for an evader at 60 miles range initially 
and a relative closing velocity of 20 knots an engagement 


with a searcher will be approximately three hours. 





Integration times of less than an hour are available with 
today's equipment. As an upper bound the case where the 
integration time is much greater than the engagement time 
may essentially be called infinite. The lower bound would 
be zero. The boundary cases are then "infinite" and "zero" 
integration time. The goal is to solve for the two optimal 
evasive trajectories and to compare them. 

The infinite integration time problem can be formulated 
to fit within the general framework of the control problem. 
This class of problems is an outgrowth of the calculus of 
variations. It is one objective of this paper to apply the 
maximum principle to the infinite integration time problem. 
As the integration time reduces to zero, the objective 
becomes to minimize the peak power. This problem is 


formulated in ordinary differential equations. 
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1.2 NOISE POWER RADIATION AND ATTENUATION 


The noise power radiation, for a line spectrum of a 


submarine has been modeled to have the mathematical form 
BQ) = eee oe Cie) 


where; a, b are constant coefficients to be determined, 
and v is velocity, measured in knots. 

Data was obtained on the radiated noise levels of 
submarines as a function of velocity at a particular line 
frequency. A least squares curve fit was made to the data. 
The form of equation (1.2.1) was used. A correlation 
coefficient, RZ, greater than .9 was obtained. This was 
considered sufficient justification for the use of equation 
(1.2.1) as a model. The power equation was then normalized 


to the forn, 
P(v) = 1.0 + b/a v4. (1292) 


The original data used for the development of the coeffi- 
cients is classified. Since the units in which power is 

is measured are unimportant as far as the analysis is con- 
cerned, and so that this paper could be unclassified, the 
coefficients were normalized. The final form of the power 


equation used was, 
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P(v) = 1.0 + 0.00005 v“. (ize) 


A further assumption is that the submarine is an iso- 
tropic radiator. That is, the effects of lobe radiation 
are not considered. The attenuation of the noise power is 
also assumed to follow an inverse square law. This was 
strictly for convenience, since the solution can easily 
accomodate any attenuation function as long as it is a 
function of range. The simplifying assumption about iso- 
tropic radiation is more serious, and as yet, no manner of 


incorporation into the proposed solution has been discovered. 
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1.3. HYPOTHETICAL SUBMARINE ENCOUNTER 


Suppose submarine E, the evader, is assigned a patrol 
area in which she is motivated to remain, but may maneuver 
at any course and speed she chooses. Suppose further that 
submarine S is searching along a straight path with a 
constant search velocity V, for E. Assume also that E is 


directly on the search path of S as in Figure 1.3.1. 





Figume 1.3.1 


In order to attempt to avoid detection, E must detect 
S and then move away. The situation where E detects S first 
is only considered here. Once S detects E, the game is over. 
The hope is that E will know about the presence of S, and 
will be able to maneuver in such a manner that minimizes 
her probability of detection. The evader is not constrained 
in this problem by acceleration, and may therefore take on 
any initial velocity desired. The question is then, what 
is the optimal trajectory which minimizes the probability 


of detection? 


Ihe: 





The nature of the solution to this problem gives the 
answer to a set of possible initial evasion states. It 
is not necessary that the evader start from a point directly 
on the path of the searcher. Evasion may begin at any state 


position. 
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2.0 PROBLEM FORMULATION 
2.1 INFINITE INTEGRATION TIME FORMULATION 


As an extreme the integration time of the searcher's 
equipment could be infinity. That is, for this situation 
the objective of the evader would be to minimize total noise 
energy received by the searcher over time. The total energy 


received by the searcher can be expressed as an integral: 


CO 
total energy received=J -| Power received (t) dt. 
. (25) 
The power received by the searcher is a function of the 
evader's velocity and distance. Assuming an inverse square 


law, 


_ (wv) 


Power Received = —5~ . (22162) 
R 


The problem may now be expressed as, 


Min 3 | PO ae, (2.1.3) 


where v and R are both functions of time. 


x5 





This class of problems, which results from the calculus 
of variations, may be solved by one of two modern approaches 
to the control problem. (2) 

Older methods of static optimization involved the intro- 
duction of Lagrange multipliers, one for each constraint; 
defining a Lagrangian expression and finding a saddle point 
of this expression. 

One modern approach to the control problem is dynamic 
programing and the application of the principle of optimality, 
which states that: 

"An optimal policy has the property that, whatever 
the initial state and decision are, the remaining 
decisions must constitute an optimal policy with 
regard to the state resulting from the first 
decision." (3) 

The dynamic programming approach basically assumes an 
optimal performance function exists and defines a recurrance 
relationship. The partial derivative of this recurrance 
relationship is called Bellman's equation, the solution 
being the optimal performance function sought. 

Another modern approach is the maximum principle. (2) 
The maximum principle can be considered as an extension of 


the method of Lagrange multipliers to dynamic optimization 
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problems. The general control problem to be solved by the 


maximum principle is; 


C1 
Max J -{ léxuee) dt + F(x,,t,) 


£0 
subject to: = = f(x,u,t) 
x(t) =X 
a (2.1.4) 
[u(t)| € U 


where I( ), F( ), and f( ) are continuously differentiable 
imc t 1 ors’; tg,%9 are parameters defining the terminal sur- 
face; and [uC t)] , the control trajectory must belong to 

the given control set U. 

In control theory there are state variables, control 
variables, and costate variables. In this problem the state 
variables, here represented by x are actually the x and y 
position coordinates, in relative space. The variables 
under "control" are here represented by u, but in the 
actual problem are velocity (v) and heading (Dp). Velocity 


and course heading are both functions of time. 


Ly) 





For each constraint a "costate" variable is introduced. 
This costate variable, Can is the dynamic equivalent of 
a Lagrange multiplier. This leads to the development of 


the Hamilitonian function which is defined to be; 
HC, u, = I(x,u,t) + £(x,u,t). (2.155) 


The infinite integration time problem may now be expressed 


as; 


Max J [ f.. dt 


x2 + y? 


subject to; & a a D 


eG) 


“ - V cos D 


x( to) 
y(to) 


initial conditions 


x(t,) 


y(t,) 


terminal conditions 


where, D is a function of time and is the course heading, 
and v, also a function of time, is the velocity. In order 
to achieve a practical mathematical solution, infinity is 


approximated with a large number. 
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The constraints dx/dt and dy/dt are obtained from the 
geometry of Figure 3.1.1. Note that the course heading, D : 


is measured from the vertical. 


y 





TERMINAL STATE 


e— — 
SS 


Be(ty).y(ep] 


ea oe a EVADER'S 
TRAJECTORY 


> 
ve, [(t) »y(to) 
INITIAL STATE 


Figure 2.1.1 - geometry relative to the searcher 


In this problem the correspondence of functions is; 


I( ) = ee oe (2.4.7) 
nee = 0 (22s) 
x aaah V sin®D C27) 
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dy . 
ll D C21. 10) 


The solution to this problem is contained in apeenai™ A. 
Just as the Lagrange Mele pineres of static problems 
yield information on the sensitivity of the solution, the 
costate variables of the maximum principle yield information 
on the sensitivity of the solution to variations in para- 
meters. 
Let J* be the optimal value of the objective function 


then 


en ee ce ae (2, aa) 


That is, the sensitivity of the optimal objective function 
to changes in the initial state variables is given by the 
corresponding initial costate variables. If a costate 
variable is small then the relative sensitivity of the 
optimal objective function will be small for small changes 
of the initial state variable. 

In static problems the Lagrange multipliers have the 
interpretation of ''shadow price" and a dynamic analogue 


exists. 
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2.1 ZERO INTEGRATION TIME FORMULATION AND SOLUTION 


As the integration time is reduced to zero, it can be 
argued that the objective of the evader is to minimize the 
peak power received by the searcher. The problem may be 
viewed as one in which there exists a threshold of power 
density level, which, if exceeded, would result in detection 
by the searcher. 

Assume there exists a set of state positions S where, 
for a given threshold of detection, the evader could not 
escape. Assume also there is another set S where the evader 
could always escape. The sets could take the form shown in 


Peeure 2.2.1. 


y So = 
~ S 
=~ 
~N escape 
| always possible 
S = 
escape = 


—. eee 


not possible 





Papure 2.2.1 


There must exist some boundary between the two sets of state 
positions. It would be of interest to know the general 


Shape of this boundary. 
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Suppose the evader is in some state as shown in Figure 
2.2.2. For a given power density threshold K there will be 


associated a corresponding velocity v, that satisfies, 


P (v1) 


a2 + y2 (2225) 


where v, is the maximum velocity the evader can have without 
exceeding the threshold of detection. Actually the evader 
is allowed to choose any velocity and heading within the 


Sercle of Figure 2.2.2. 


x 


(x,y) 
state position 


Faiguine. 2 s257 


The maximum velocity possible and the state position deter- 


mine the angle ©,. That is, 


Q, = are sin (v,/V 5) (239092) 


By geometry D = @,. The question the evader must answer is, 


which controls should be chosen. 


Ze 





Suppose the evader is in set S, the set of states where 
he cannot escape detection no matter which controls are 
chosen. In Figure 2.2.3 it can be seen that the shape of 
the boundary must be such that its slope at the state 


position is greater than or equal to ®,. 





Figure 2.2.3 - boundary with slope < 6, 


If the slope of the boundary were less than ©; it would be 
possible for the evader to escape from the set S. This 
would be a contradiction. 

Suppose the evader is in the set of states where he 
can always escape, provided he chooses the right control. 
Since the evader is not allowed to choose a control which 
produces a © greater than Or the slope of the boundary 
must be less than or equal to 6. Otherwise E would always 
move from the region of escape to non-escape as shown in 


Figure 2.2.4. 
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Figure 2.2.4 - boundary with slope greater than 0, 


In order to resolve these contradictions, the evader 
should choose the controls which produce a ©, which is 


tangent to the boundary. This is shown in Figure 2.2.5. 





Figure 2.2.5 - boundary with slope equal to 6, 


As shown by Figure 2.2.2 this ©, is the result of choosing 
the maximum velocity v,. 

Since the choice of a velocity control will be always 
to maximize velocity for a given power level which means to 
always produce as much power as possible, the result is that 


the power level will remain constant at the detector. 
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For the zero integration time situation the objective 
is to maintain a constant power level received by the 


searcher. This may be expressed as 


PGi) 
PFINAL = as: = K (22223) 
X£ a Vie 
where K is a constant, and the £ subscript indicates the 
final value at the termination of the trajectory. 


The equations of motion may now be derived from the 


geometry as shown in Figure 2.2.6. 





Figure 2,250 - 


Velocity, v, and heading, PD , are functions of time. The 


equations of motion are; 


ax ; 

Wee = oe oe sin (2.2.4) 
d - 

Si = ¥ cos D (2.275) 


2) 





Equations (2.2.4) and (2.2.5) may be combined to express 


dy S v cos P (2.2.6) 
dx -V, + vin ® 


The problem now arises that the initial conditions 
which would allow a solution to the equations of motion are 
part of the results that are sought. One question that the 
evader would have, is what initial velocity and heading 
should be chosen. Further analysis of the nature of the 
trajectory should supply a solution. 


The following remarks refer to Figure 2.2.7. 





Figure 2.2.7 


Suppose the evader is in state (XQ Yo) and has a given 
power level, K. This implies that v},(0) is greater than 


zero, where v;(0) is the velocity at state (x9,Yo)- There 
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will be associated with this power level, K, a trajectory, 
not necessarily optimal. 

Suppose the final velocity at x=0 is greater than zero. 
The game is considered over at x=0 since this would be the 
minimum range, and hence, the evader could simply stop all 
action and continue to improve his situation. It is con- 
tended that there exists a K' less than K, since a v,1(0) 
could be chosen which is less than v,(0). This would imply 


that 


P (v,1(0)) P (v,(0)) 


2 2 2 
Xo -G 


5 (2.297) 
7 
Since v1 (0) v},(0) , this implies that 6,:<®,. Initially, 
at least, the K' trajectory will lie below the K trajectory. 
Could K' cross K as indicated in the figure? If this were 
to occur, the 6,1 would be greater than 9,, which would imply 
that K' is greater than K. This is a contradiction, and 
Before. the trajectory for power level K' cannot cross 
that for K, but must lie below it. 

Since trajectory K' is below that for K, the terminating 
distance y,;1 must be less than y,. From equation (2.2.3) ‘the 
conclusion is that Ver must be less than Ve: That is, the 


terminal velocity must decrease. The result then is to 


ag 





choose a lower power level until the terminal velocity is 
zero. 

Could the terminal velocity go to zero before reaching 
the y axis? This is not a feasible result, since this is 
not the state position of minimum range. Due to the move- 
ment of the searcher the range would decrease and the power 
received would increase above the detection threshold. 

The object was to discover the initial conditions so 
the equations of motion could be solved. Instead the 
termination conditions are known. These will be used as 
initial conditions by reversing time. The system of equa- 
tions to be solved is now; 


axe 7 : 
ile Ve > Vo sie D (25 2a3) 


sy - Vv cos D (2.2.9) 


and, 


x(t) = 0 


y(to) parameter 


v(to) 0 


P (to) = 0 


are the initial conditions. 
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3.0 RESULTS 


The results of the analysis consist of giving advice 
on the choice of control variables to an evader. Both 
cases are presented; infinite integration time in graph 
(1.A4.) and zero integration time in graph (1.B.) Advice 
is given in both cases only for the first quadrant, as this 
is the region of greatest concern. Four evasive trajectories 
were calculated and are shown on each graph. Those trajec- 
tories considered were for initial evasion ranges of 60, 
40, 30, and 20 miles. Trajectories were calculated for 
each of the four initial ranges both for infinite and zero 
integration time. The set of graphs labeled 2.-- are for 
a search velocity of 15 knots. Set 3.-- is a replication 
of set 2.-- but for a search velocity of 10 knots. 

For the 10 knot search velocity and infinite integra- 
tion time trajectories were calculated for initial evasion 
ranges of 30 and 20 miles. The 60 and 40 mile curves went 
off scale and were omitted. 

Control variable measurements were taken from the 
velocity and heats graphs and these control vectors were 


plotted on the relative movement plots in graphs 1.A and 
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1.B. These vectors are the advice which would be given to 
an evader. It is not necessary that an evader start at a 
state position such that he is directly on the path of the 
searcher. He may start at any position and begin evasion 
with the control vector belonging to that state. If the 
searcher should change speed it would be necessary for the 
evader to consult a new set of control variables and continue 
the evasion. 

Graph 1.C Ane a comparison of the relative movement 
plots of infinite and zero integration times. Only the 60 
and 30 mile curves are presented for clarity. The control 
vectors for the two situations do not appear significantly 
different during the initial stages of the trajectories, 
and the result is similar trajectories. As the range de- 
creases the effects of integration time become more apparent. 
The graph shows that near the minimum ranges the effects of 
integration time are critical and hence it would be of 
interest to know what the integration time of the detector 
is. During the initial stages of evasion knowledge of the 
integration time is not critical. 

Graph 1.D is a representative plot of the costate 
variables. The costate variables, in part, represent the 


importance of their corresponding state variable. Costate 
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variable Ark corresponds to the x coordinate and AX 5 to the 
y coordinate. The graph shows that the evader, starting at 
State position (x = 60, y = 0) wants to increase y and 
values y slightly over x. That is, the evader values an 
increase in the lateral distance, y over an increase in 


time. This is true throughout the trajectory. 
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The curves are for the initial 
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APPENDIX A 


SOLUTION TO INFINITE INTEGRATION TIME PROBLEM 


The problem may be stated as, 


te 
1 
MAX J -| Se 
me Py 
£9 
subject to; - 
d 
ae mii ued aa) D 
ays 
ar v cos D 
ylty) = yo 
Ce) a 
where P(v) = 1. + 0.00005 vt: Velocity, v, and course 


heading, PD , are functions of time. Velocity is measured 
in knots. The limits of integration should, in theory, be | ‘ 
As stated ty = 0 and infinity is approximated with a large 


number t,. The symbols are as shown in Figure l. 
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(Xo oe 
Figure l 


The Hamiltonian function is now defined as 


H CRO: i al (x,y,v,@,t) a8 A £ (x,y,v,D,t) 


In the case under discussion, 


Gx,y,v,@,t) = ae F Ay CY saeey sin® ) ae Vv cos) 
(1) 
The object is to maximize the Hamiltonian function at 
every point of time along the optimal trajectory by choice 


of the control variables, v and QD. It is required that; 


oh. 








Ae 





Ov , oO x 
O° 5) OH Ga 
H 0 ; =O 
aD 2 OY 
me Ct, Oo Caw 





New, from (1) and (2), 


_ 2 x P(v) 

1 (2 Bere B22 ey 
5 2 y Be 
Ae = Ge yh? " 


Also, 





ou ; ee Be ie sin + Ms, cos D (5) 


x y 


< 


evita A, v cosh - A, v sin® (6) 


Setting equation (6) equal zero implies, 


Ay > sin 
Ao cos PD 
or, a 
@ sta OR (7) 


From trigonometry and Figure 2 it can be seen that, 


A» 
cos — (8) 
. CA tee 
r 
and sin®@ zs (9) 


( Aq? 4+ no2 )% 
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Figure 2 


Substitution of equations (8) and (9) into (5) results in; 
piv) | An? 4 Ad? fl s/ 12. doe 
x2 + y2 (12 + 2,2)? 1° + A2 


Therefore, P'(v) = (x? + y*) A,2 +A22 (10) 


Brace P (v) = 1.0 + 0.00005 v", 


Pile) Sono - 


Equating (11) and (10) and solving for v results in 


v = (5000. (x2 + y) VE Ay? + Neg o- (Gly) 


Equations (7) and (12) are the conditions for optimality, 
and must be satisfied at every point in time along the 
optimal trajectory. 


Since X,(t,) = Ap(t,) = 0, from (12) it follows that 


v(t) =O . (13) 


oy, 





A4(ty) 
TOG 


form a solution is sought in L'Hopital's rule. Rewriting (7), 


Since Ges) = tan7? | 


| is an indeterminate 


and applying L'Hopital's rule 


lim tan D (t,) = Ce (14) 


t>t} oO Aoft1) 
Ot 


Substituting (3) and (4) into (14) and canceling similar 


berms results in 


Lim anaes) <= Soom yuee (15) 
IL D, 
t= & 
i 
/ 
Rewriting (15) the terminal condition for the control 
variable ‘) is 
D Ctpae= rene — 4 (16) 
I y 
The objective is to find the values of the control variables 
at every point in time. Of principle interest are the 
initial velocity and heading. The solution now involves 


four simultaneous partial differential equations. The 
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initial conditions of variables x and y could easily be 
given, but only the terminal conditions are known for 
Ay and A> The solution is to reverse time and call 
the initial conditions terminal conditions and vice versa. 
Reversing time changes the signs of the differential equa- 
tions, but has no effect on the optimal conditions. 
The problem solution in backward time may now be 


stated as, 


: V - v sin D 
xX Ss 


-v cos PD 


oO _ 22 xXe eG 
(x2 + y2)? 


O = 2y0kGp 


where the "o'' represents backward time. The initial 


conditions are; 


x(t) parameter 


State Variables 


Cee) parameter 


SP) 





A 1 (to) = 0 
costate variables 


Ao(t 9) = 0 


v(t.) =" 
control variables 


D(t,) = tan *(x/y) 
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APPENDIX B 


COMPUTER OUTPUT AND METHODS OF SOLUTION 


The computer routines given in this appendix were set 
to operate on the time sharing system (cp/cms) of an IBM 
360 computer. The graphical display package of a Tektronix 
4012 display terminal were used. Various termination state 
positions were attempted until an initial evasion range was 
selected. Since all the information for a particular 
trajectory was calculated at each execution, it was only 
necessary to change the call statements in the plotting 


package for different visual display. 


oy 





FLOW CHART FOR INFINITE TIME SOLUTION 


INITIAL CONDITIONS 


ee 
ae Tt is am hours 
Ay =0 > v is in knots 
Az =0 OE ENS x,y are in miles 
en & ; we is searchers speed 
@ = tan” X/YDCONTROL in SINE 
P(v) = 1+ bv 









DIFFERENTIAL EQUATIONS 


XeV ev cos 6 
Y= ev sin 

Qo } ‘ 
R= 2X P(v)/R 
Ay= 2 ¥ P(v)/R* 








OPTIMAL CONDITIONS 


P'(v)=R* V oar ». 


(gives solution for v) 


) = tan! (Ni/ \) 
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